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ABSTRACT 

An evaluation model is presented for educational 
decision makers with evaluation defined as "the process of 
ascertaining the decision areas of concern, selecting appropriate 
information, and collecting and analyzing information in order to 
report summary data useful to decision-makers in selecting among 
alternatives*" Areas in special need of evaluation include systems 
assessment of the range and specificity of educational objectives, 
program planning, program implementation, program improvement, and 
program certification or generalizability to other situations* Two 
feedback loops are explained to distinguish particular evaluation and 
decision , elements for educational systems and instructional programs* 
This model is compared with the Corrigan-Kauf man six-step 
problem-solving model* (JK) 
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This paper attempts to discuss a basic model for the evaluation of educational 
systems and instructional programs. The assumptions leading to the develop- 
ment of a comprehensive definition of evaluation and the definition Itself will 
be presented. A discussion of this definition leads to the description of a 
decision-oriented model for educational evaluation. This model Is compared 
briefly with the Corrlgan-Kaufman Si>'-Step Problem Solving Model, The paper 
concludes with a discussion of the current research activities of the Center 

for the Study of Evaluation in regard to their relation to the basic evaluation 

model . 

IMTRODOCTION 

Inherent in the development of a model of educational evaluation is the develop- 
ment of a theory of evaluation. The development of such a theory of evaluation 
has been established as the major goal of the Center for the Study of Evaluation. 
At their best the propositions presented In a theory of evaluation should enable 
one to predict, fully, the appropriateness of utilizing various evaluation 
strategies within a system. It is granted that what has been developed is not 
an evaluation theory but rather a conceptual framework. What is presented in ■ 
this paper represents months of conceptual efforts and may be thought of as a 
first approximation of an attempt to develop an evaluation theory. In other 
words, a rationale has been constructed for conducting evaluations in a certain 
way that is based on a specified set of assumptions which in turn underlie a 
precise definition of what an evaluation is supposed to be. Development of aa 
evaluation theory is thus and ”end” rather than a means, guiding the research 
activities of the Center for the Study of Evaluation. 



evaluation presented here is based on the following assumptions: 

Evaluation is a process of gathering information. Most past definitions 
of evaluation are inadequate since they do not cover the full range of 
I activities requiring information. 

2, The information collected ia an evaluation will be used mainly to make 
decisions about alternative courses of action, rather than being 
employed in some other fashion. Thus, the manner in which the infer- 
|v , mation is collected, as well as the analysis procedures, must be 

I appropriate to the needs of the decislon-makar or of potential declsion- 

I ' . . involved publics. This requirement might necessitate quite different 

analyses than that which might be employed if the purpose were under- 
I standing, the education process per se. 

i . ■ * 

I 3« Evaluation information should be presented to the decision-maker in a 

form that he can use effectively and which is designed to help rather 
than confuse or mislead him. 
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4. Different kinds of decisions may require different kinds of evaluation 
procedures. 

These assumptions have led us to a broader more comprehensive definition of 
evaluation. . Past definitions of evaluation have either equated it with; (1) 
measurement and testingj (2) statements of congruence between performance and 
objectives, or (3) professional judgments. None of these definitions by itself 
is sufficient to provide all the necessary information or to Include the multi- 
plicity of activities now regarded as evaluation. 

The Center feels an expanded view of evaluation is necessary in order to take 
into consideration the idea that the Judgments from evaluators are intended to 
be of use to decision-miakers in their process of selecting between various 
courses of action. This view of evaluation also acknowledges the uniqueness of 
specific situations or programs and the necessity of recognizing this uniqueness 
In the evaluation as well as in the manner in which the evaluation information 
is ultimately reported. While there are any number of variations of a specific 
wording that might serve equally well for a definition of evaluation, one has 
been devised which fits the Center's conceptions of evaluation and meets its 
biases. We would maintain that evaluation must take into consideration the 
ultimate decision-making functions to be served, as well as the nature of the 
specific problem or situation under analysis. The following definition is 
preferred: 

Evaluation ijs the process of ascertaining the decision areas of 
concern , selecting appropriate information , and collecting and 
analyzing information in order to report summary data useful to 
de c 1 8 ion-make rs in selecting among alternatives . 

The first part of the definition of evaluation presented here deals with ascer- 
taining the decision areas of concern. The decision-maker, and not the evalua- 
tor, determines the nature of the domain to be exasrdned. The evaluator can and 
should, however, point out inconsistencies, potential difficulties, or additional 
data that might modify, the decision-maker's views on the relevance of certain 
concerns. 

For example, if the evaluator is called upon by a specific decision-maker to 
provide an evaluation, he will first want to know what should be evaluated. 
Decision areas of concern may be. stated relative to explicit statements of 
goals or objectives of the system or relative to various implicit goals. In 
his interactions with the decision-maker, the evaluator may wish to point out 
the necessity for broadening the area of concern because of interrelated aspects 
of the school program, or to consider, as well, various areas of potential un- 
anticipated outcomes. 

On the other hand, if the evaluator is conducting an evaluative study of an 
educational institution without having been commissioned by a specific decision- 
maker, he has available greater flexibility. A principal, for, example, might 
make a decision on his own to conduct an evaluative study of his school. There 
is a preconception on the part of the evaluator as to which decision-maker or 
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potential decision groups he is directing his work towards. Thus, the decision 
area of concern In such an endeavor la framed by the unique nature of the 
potential decision-maker or decision groups along with either actual data or 
.'Judgments on the part of the evaluator as to the concerns of this group 
(individual) . 

The Center considers this "'preconception of dec is ion-maker" notion a fundamental 
and useful distinction between evaluation and some kinds of research. If one 
realizes that the purpose of what he does is to provide the best possible basis 
for Informed Judgraents or decisions, his thinking about his task will surely be 
influenced; and this will be a different influence than that which operates on 
the researcher whose purpose Is to discover or explain some phenomenon. 

Another part of the definition and, therefore, another task of evaluation, deals 
with selecting appropriate information in light of the decision areas to be 
considered. If the decision area relates to the assessment of the needs of a 
total system, the information requirements will be quit€i different than when 
the decision area is related to the relative success of two specific alternative 
programs conducted under experimental conditions. The task of evaluator in 
specifying information requirements includes the development of the evaluation 
design .of the project, and the selection and/or development of instruments 
designed to provide the information appropriate to the decision areas. 

Collecting and analyzing the information are tasks of prinue concern to the 
evaluator. He will encounter different problems associated with these tasks, 
depending upon , the ■ unit being evaluated, the nature of the decision-maker and 
other considerations. 
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One of the most vital parts of the evaluation process is repor ti ng summary data 
to the decision-maker. Most evaluators often overlook this function as being 
merely a perfunctory exercise. The evaluator’s role requires that he make Judg- 
ments about the relative worth of various courses of action. These Judgments 
may be in the form, of statements or recommendations to the deci s ion-maker (s) , 
or may be general descriptive material. But in all instances the evaluator 
should attempt to be explicit in .the specification of the value system that led 
to the judgments made. Indeed, if the purpose of evaluation is to provide 
information that will enable decision-makers to reach decisions about alterna- 
tives, then the nature and form of the reporting should be appropriate to the 
problem and the audience. 

The summary data is provided to be of use to the decision-maker . It has already 
been alluded to in this paper that the term "decision-maker" is used to apply 
both to an explicit contractor of evaluation services as well as a potential 
bufonly implicit decision-maker or group. Moreo.ver, the term "decision-maker" 
is •used to apply both to an individual with organizational "line" authority 
(e.g., a school principal) as well as to other people that participate in '%he 
decision process or in the development of educational policy decisions. Through' 
out this paper, whenever. we refer to "decision-maker" it is in the generic sense 
discussed above. 
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Iiiforaiatlon is provided to "decision-makers,*'' in order to enable sounder 
decisions in selecting among; alternatives . By definition, a "decision** involves 
making a choice among alternatives. However, the form of alternatives has a 
wide range. Alternatives may range from a **go no-go’* category regarding a given 
textbook for a particular classroom to a cojuplex aggregation of a number of 
budget categories related to an optimum expenditure level. In general, the number 
of categories of alternatives increases as the size of the program or sysitem 
increases; e.g., pupil achievement, teacher morale, teacher practices, etc. 

Tfie summary evaluation data should ordinarily be presented in the form of state- 
mcnts and/or recommendations about alternatives. An exception would be when 
such info.rmation is designed to describe the status (past, present, or future) 
of the system. For example, information that Indicates students in a certain 
school are weak in mathematics. In this Instance, there are no alternatives and 
the decisions are implied (e.g., something should be done to correct this situa- 
tion) . 

THE FIVE EVALUATION NEED AREAS 

The foregoing definition and assumptions are closely tied to the decision-making 
process, which in turn leads to a consideration of what k inds of educational 
decisions require evaluative information. Inquiry along these lines has led to 
'the development of a decision-oriented classification of the various types of 
evaluation. Five kinds or need areas of evaluation may be identified, and each 
is designed to provide and report information useful to a decision-maker in 
making Judgments relative to decision categories. In other words, there are 
evaluations necessary in making decisions about (a) systems assessment, (b) pro- 
gram planning, (c) program implementation, (d) program improvement, and (e) 
program certification. Thinking related to the development and specification 
of these stages has drawn heavily from the work of Hemphill, Lumsdaine, Provus, 
Stufflebeam, and Skager. 

Systems Assessment 

Systems assessment is a means of detemining the range and specificity of educa- 
tional objectives appropriate for a particular situation. The needs may be 
represented as a gap between the goal and the present state of affairs. The 
evaluative problem then becomes one of assessing the needs of students, of the 
community, and of society in relation to the existing situation. Assessment, 
therefore, is a statement of the status of the system as it presently exists in 
comparison to desired outputs or stated needs of the system. 

A systems assessment might be related to evaluation of a specific instructional 
program and thus the charge would be to determine the present status relative 
only to a specific objective and related objectives. This could be referred to 
as a "sub-system assessment." 

Program Hanning 

Progrsim planning, the second need area, is concerned with providing information 
which will enable the decision-maker to make planning decisions -- to select 



between elternatlve processes in order to make a judgment am to which of them 
ahould be introduced into the system in order to fill most efficiently the 
critical needs previously deterrained. In an instance where we are proceeding 
through severe need areas in sequential fashion, the fol Imping might occur. 

After the decision-maker receives the systems assessment evaluation, he might 
make a decision as to the appropriate means of fulfilling that need. Alter- ; 
natively, he might designate several prosslbllities and afik the evaluator to 
provide Information on the possible Impact of each. Henc®, in program planning, 
the evaluator provides the data for an evaluation of a program prior to its 
inception. The task of the evaluator is to anticipate the attainment of goals 
and to assess the potential relative effectiveness of different courses of . • I 
action. I j 

Program Implementation 

After the decision-maker has selected the program to be implemented, an evaluation 
of program implementation determines the extent to w'hlch the implemented program 
meets the description formulated in the program planning decision. In the case of 
an existing program where no known changes have been Implemented, the evaluation 
task at this stage is to determine the degree to^which planning descriptions c£ the 
program^ coincide with the Implemented program and the extent to which assumed 

descriptions of inputs to the system (e.g., students) corresponds with observed 
inputs. 

Program Improvement 

The evaluator can play an important role in Program Improvement, the fourth 
need area, by providing as much information as possible about the relative 
success of the parts of the program. In order to perform program improvement 
evaluation, it is necessary to recognize the basically interventionist role 
that the evaluator has been asked to take. It is quite natural for the decision- 
maker to expect that as the evaluator identifies problems and collects and 
analyzes related information while the program is in operation,: data relating 
to possible changes that could be executed within the systiem to improve the 
program will be presented to the decision-maker Immediately. 

This need area has often been overlooked or ignored by the traditional evaluator 
who has attempted to impose the antiseptic sterility of the laboratory on the 
real world. Such an approach may make for a fine experiment, but it does 
little to improve a program, which is often not in its final fomm. 

Program Certification 

Finally, the fifth area of evaluation, Program Certification, must provide 
the decision-maker with Information that will enable him to make decisions about 
the program as a whole and its potential generalizabllity to other situations. 

The evaluator might attempt to provide information which will enable the 
decision-maker to determine whether the program should be eliminated, modified, 
retained, or introduced more widely. 
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Tbe kind of information collected for program certification decisions is in 
large part dependent upon who is the intended decision-maker. It is obvious 
that different information will be required if the potential decision-maker 
is the teacher » the principal, or a funding agency. Evaluations in this area 
will be concerned with examining the extent to which the objectives have 
been achieved, as well as with the impact on the outcomes of other programs. 

In program certification evaluations, there is a requirement for valid and 
reliable data which would generally require that the evaluator attempt to 
apply as rigid a set of controls as possible. The evaluator might use pre- 
and post-test designs and employ sophisticated methods for analyzing the 
data. Intervention should be avoided in evaluations in this need area. 

Here the traditional evaluator is ’’at home." 

EVALUATION SYSTEMS AKB PROGRA>IS 

The Center has made a distinction between the evaluation of educational systems 
and the evaluation of instructional programs. In terms of the conceptual 
framework that has been presented, one can view the evaluation of instructional 
programs as involving the last three evaluation areas. 

In evaluating any educational system it is necessary to determine the educational 
needs in terms of the most appropriate objectives for the given system and to 
devise a procedure for providing regular information on the progress of the 
system relative to these dimensions. This procedure is the evaluative device 
for decision-making about the assessment of system needs (Systems Assess- 
ment). When decisions have been made about the objectives of the system 
which are Inadquately met, the decision-maker might then be concerned with 
the selection of programs to meet these objectives. Evaluation information 
might be sought relative to the possible impact of various courses of 
action or programs (Program Planning). 

Thus, if one followed through on the full cycle of evaluation in an 
educational system, including the allowance of feedback and recycling, the 
process might be depicted as In Figure 1. 

1 

(See Figure 1 next page) 
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The evaluatloa of -an lisa t rue tioaal program assumes the prior assessment of the 
program or of a larger aystem, a declaion about objectlvea to be attended to, 
and the selectios of programs considered to be appropriate for nee ting these 
objectives. That is, the evaluation of an Instructional program ordinarily 
begins after the decisions related to need areas I- and 2 of the evaluation 
have been made. In evaluating an Instructional, program, the objectives to be 
achieved and the program which it is assumed will be most successful in 
achieving these objectives are generally considered as "’given.”® Thus, the 
evaluation of an instructional program focuses primarily on the last three 
need areas of evaluation. 

Where the evaluation task commences with the evaluation of the instructional 
program, the neceaslty is envisaged for a sub-system assessment dealing with 
the area of concern of the selected instructional program. Thus, It is seen 
that the evaluation needs areas are not necessarily setjuentlal with the steps 
easily defined. In some instances, moreover, the data collection, analysis, 
and reporting appropriate to a decision might be so easy to obtain or so 
inextricably tied to the making of the decision that the decision-maker and 
hia staff would perform the evaluation thesnj?elves. In some instances, the 
project begins for the evaluator after a number of decisions have already been 
made. Thus, the evaluator might have to attend to only selected evaluation 
need areas. 

For the sake of convenience, Figure 2 depicts a way In which the evaluation 
need arisas might be interrelated in the evaluation of instructional programs. 

A final explanatory note is in order concerning the role of the evaluator in 
this evaluation rtodel. It might be possible to draw the conclusion that the 
evaluator does all things -- that he Is curriculum designer, administrator, 
program implementor, test officer, budget manager, etc. This is a misconception. 
This notion has been partially dispelled by our commeiit earlier that what has 
been described in this section is the full range of the evaluation cycle. 
Functions to be performed are being described rather than a role in each 
evaluation need area for a specific individual. 

In an instructional program evaluation the evaluator la, first, concerned with 
providing information on progrsm Implementation and for program Improvement. 

The appropriate information is related to the extent to which the program has 
been linplemented in a manner compatible with that described in the program plan, 
and the extent to which the content described in that program design was por- 
trayed accurately. In addition, the evaluator ia concerned with providing 
information which may be used in modifying and improving the program. The 
decision-maker may wish to recycle areas 3 and 4 of the model any number of 
times in order to Insure that benefits from program Implementation and improve- 
ment are maximised. 

COMMBOTS RSGARDIHG THE C03RIGASI-1CATUIMAK FROBISM SOLVIHG MOOSL 

Kauj&aan suggests that the six-step problem solving modal as developed by Corri- 
gan and Kaufman is an appropriate model for describing the educational management 
process and that' it nay be used as, a referrent for identifying the various need 
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ureas In the evalaistlve process. Both models belong to the family of systems 
, approaches to education. Both models represent atteisapts to provide process 
nodele for educators to aid them in structuring their thrusts to%rard the goal 
of quantitative improvement of education. 

The Corrigan- Keu&^an model is presented as a comprehensive approach for the 
use of the. educational manager or decision-maker. The evaluation model dis- 
cussed iS' designed primarily for the use of the professional educational 
evaluator. This individual (or group) may or may not be the decision-maker 
as 'vas pointed out earlier in the paper. The two models would appear to be 
eoDiplimentary and one could envision both models being ecqiloyed in the same 
educational situation. 

It should be noted that in the past, evaluation has been seen as a process that 
occurs only toward the end of a problem-solving model (such as the "’'Determine 
Performance Effectiveness"" function in the Corrigan- Kaufman model). The case 
has been presented in this paper that evaluation needs to be considered as a 
comprehensive process that applies to all areas of decision-making in education. 

If evaluation is concerned with providing Information to the decision-maker 
that will help him to effectively choose between alternatives (as is argued in 
this paper), then this information is needed throughout the management process, 
not Just at the end. Evaluation should begin when an educational agency first 
looks, critically at the potential of the educational process in its particular 
system, not after all attempts at improvement have been concluded. 

TEE CEOTER FOR T!ffE STUDT OF EVAmilONI 

The Center for the Study of Evaluation, one of nine USOE E & D Centers, is a 
unique organization in that its efforts are devoted exclusively to the study 
of the evaluation of educational systems and instructional programs. After an' 
initial period of exploration, the Center^ s activities have been increasingly 
focused on a relatively few research projects which 'fall within the scope of 
major program areas. This heavily programoiatic approach to the search for a 
meaningful evaluation framework has led to the development of two major 
programs; a program on Evaluation on Instructional Programs and a program on 
Evaluation of Educational Systems. The Center is supporting one major project 
in the first program and two in the second. 

In response to the kind of needs of the program on Evaluation of Instructional 
Programs, the Center has established the Project for Research of Objective- 
Based Evaluation (PROBE). This project has as its purpose the specification 
of behaviorally defined objectives across a broad range of subject areas and 
the development' of test items related to these objectives. Further, the project 
will study the format of the objectives and devise procedures for their intro- 
duction which have the greatest impact upon the instructional programs and their 
outcomes. Products derived from this project will be useful to evaluators work- 
ing in areas 3, 4 and 5 of the evaluation model. 

TWO current Center projects at the GSE at UCLA, the Elementary School Evaluation 
System and the Higher Education Evaluation Project, represent attempts at 
examining more complex educational systems (areas 1 and Z of the model). Each 
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is designed to yield procedures for information collection analysis and 
reporting and to assist educational evaluators and decision-makers. 




The Elcroentary School Evaluation Project is developtSg a ®®do-it-yourself®" 
evaluation kit to help elementary school principals assess student perfomances 
across a vide range of concepts and skills, to interpret the results of this 
information, and to make valid decisions for iisproving performance levels. 
Included in the kit will be an ’’’atlas of Norms®® from which students® performance 
can be compared with that of other schools sharing similar factors, such as 
the socioeconomic status of the coismunlty, the region of the country, and the 
community® s degree of urbanization. 







The Higher Education Evaluation Project is developing a vide variety of measure- 
nent tools which administrators can use to evaluate both the long and short 
term effects of their institutions upon their students. At present the project 
is conducting a survey of students and alumni at more than 75 diverse colleges 
and universities located throughout the United States. The questionnaires used 
in the survey and a manual eicplaining their use and the various ways of inter- 
preting the results will be made available to any institution interested in 
evaluating its own higher education program. 



Thus, the products of these three major projects of the Center -- PROBE, the 
Elementary School Evaluation System, and the Higher Education Evaluation 
Project -- are designed to be of value in information collection analysis and 
in reporting within various dimensions of the five types of evaluation. 




